Hierarchical Markov Random Fields with Irregular Pyramids for Improving Image Annotation

Image segmentation and Automatic Image Annotation (AIA) are two important areas that still impose challenging problems. Addressing both problems simultaneously may improve their results since they are interdependent. In this paper we give a step ahead in that direction considering different segmentation levels simultaneously and possible contextual relations among segments in order to improve the automatic image annotation. We propose to include hierarchical relations among regions of an image in a Markov Random Field (MRF) model for annotation. This relations are obtained from irregular pyramids, which keep parent-child relations among regions through all the levels. Our main contribution is therefore the combination of the irregular pyramid approach with context modeling by means of hierarchical MRFs. Experiments run in a subset of the Corel image collection showed a relevant improvement in the annotation accuracy.